18:42
2026-07-16
twitter.com
machine-learning
Large Open-Weight MoEs Are Converging on Lower Expert Activation Ratios
Analysis of six large open-weight mixture-of-experts (MoE) models from 2025 and 2026 shows the average routed-expert activation ratio dropped from 3.7% to 2.1%, indicating a trend toward larger expert…